1. Field of the Invention
The present invention relates to a super-resolution observation apparatus that can obtain a sample image having a super-resolution exceeding a resolution limit of an image forming optical system.
2. Description of the Related Art
In recent years, techniques of obtaining a sample image having a resolution exceeding a resolution limit of an image forming optical system (hereinafter referred to as a super-resolution) have been developed and put into practical use. For example, a microscopy called SIM (Structured Illumination Microscopy) is known as one of such super-resolution techniques. SIM is disclosed, for example, by International Publication Pamphlet No. 2006/109448, and M. Gustafsson, et. al., Biophysical Journal, Vol. 94, pp. 4957-4970.
In a microscopic observation, illumination light is irradiated on a sample as uniformly as possible with a general wide-field observation method. With SIM, however, illumination light is modulated, and illumination light with fringes is mainly irradiated on a sample. As a result, a spatial frequency of an observation light intensity distribution contributing to image forming can be shifted. By restoring the shifted spatial frequency to the original state after the illumination light passes through the image forming optical system, a sample image having a super-resolution exceeding a resolution limit of the image forming optical system can be generated.
FIG. 1 illustrates a basic configuration of a conventional super-resolution observation apparatus. FIG. 2A illustrates an observation light intensity spatial frequency characteristic on a sample S. FIG. 2B illustrates an observation light intensity spatial frequency characteristic at an image position I. FIG. 2C illustrates a spatial frequency characteristic of a super-resolution image for which a super-resolution process has been executed. A basic configuration and operations of the conventional super-resolution observation apparatus are described with reference to these figures.
The super-resolution observation apparatus (1) illustrated in FIG. 1 is configured by including: an excitation light irradiation unit (2) for irradiating, on a sample S, excitation light intended to excite a label included in the sample S and to emit observation light; an excitation light modulation unit (3) for modulating a spatial intensity distribution of the excitation light on the sample S; an enlarged image forming unit (4) for forming an enlarged image of the sample S at the image position I from the observation light generated by irradiating the excitation light on the sample S; an image capturing unit (5) for converting a spatial intensity distribution of the enlarged image into digital image data; and a super-resolution processing unit (6) for generating a super-resolution image having a super-resolution frequency component exceeding a cutoff frequency of the enlarged image forming unit (4) from one or a plurality of pieces of the digital image data.
The excitation light irradiating unit (2) can irradiate, on the sample S, excitation light having a wavelength band that can excite the sample S, and a sufficient intensity. A label for locally generating observation light having an intensity almost proportional to the intensity of the excitation light is distributed on the sample S, and observation light having a spatial intensity distribution that correlates with a product of the spatial intensity distribution of the excitation light on the sample S and a spatial concentration distribution of the label is generated from the sample S. If the intensity of the excitation light on the sample S is uniform as typified by a wide-field observation method, the spatial intensity distribution of the observation light on the sample S is similar to the spatial concentration distribution of the label. Also an observation light intensity spatial frequency characteristic (7) obtained by performing a Fourier transform on the spatial intensity distribution of the observation light is equal to a label concentration spatial frequency characteristic (8) obtained by performing a Fourier transform on the spatial concentration distribution of the label. In contrast, if the intensity of the excitation light on the sample S is not uniform, the spatial intensity distribution of the observation light on the sample S is different from the spatial concentration distribution of the label. As illustrated in FIG. 2A, some shift components (9) obtained by shifting the label concentration spatial frequency characteristic (8) by a shift amount fs in a frequency space are added to the observation light intensity spatial frequency characteristic (7) with a certain linear combination coefficient.
The excitation light modulation unit (3) is arranged between the excitation light irradiation unit (2) and the sample S, or included in the excitation light irradiation unit (2). The excitation light modulation unit (3) can change the spatial intensity distribution of the excitation light on the sample with time. Accordingly, the linear combination coefficient of the shift components (9) of the label concentration spatial frequency characteristic (8), which are included in the observation light intensity spatial frequency characteristic (7), can be changed.
The enlarged image forming unit (4) projects the spatial intensity distribution of the observation light on the sample S at the image position I. The enlarged image forming unit (4) has a unique cutoff frequency fc, and cannot project a portion of spatial frequencies having an absolute value that is higher than fc in the observation light intensity spatial frequency characteristic (7) on the sample S at the image position I. This means that the observation light intensity spatial frequency characteristic (7′) at the image position I includes only a portion of spatial frequencies having an absolute value that is lower than fc in the observation light intensity spatial frequency characteristic (7) on the sample S as illustrated in FIG. 2B. Accordingly, fc determines a resolution limit of the enlarged image forming unit itself.
The image capturing unit (5) converts the spatial intensity distribution of the observation light at the image position I into digital image data composed of brightness values. The brightness values included in the digital image data are associated with coordinates of the image position I, and have a value almost proportional to an intensity of the observation light at each coordinate position. If the spatial intensity distribution of the excitation light on the sample S is uniform, this digital image data results in an image composed of a portion of spatial frequencies having an absolute value that is lower than fc in the label concentration spatial frequency characteristic (8). This image is called a wide-field image. A cutoff frequency of the wide-field image is fc that is the same as the cutoff frequency of the enlarged image forming unit (4). In contrast, if the spatial intensity distribution of the excitation light on the sample S is not uniform, the digital image data includes also portions of spatial frequencies having an absolute value that is lower than fc in the shift components (9) of the label concentration spatial frequency characteristic (8) as illustrated in FIG. 2B.
The super-resolution processing unit (6) generates a super-resolution image composed of components including up to spatial frequencies fc+fs having an absolute value that is higher than fc in the label concentration spatial frequency characteristic (8) by restoring the shift components (9) of the label concentration spatial frequency characteristic (8), which are included in the digital image data generated by the image capturing unit (5), to the original frequencies by the shift amount fs as illustrated in FIG. 2C.
Influences that a noise component included in digital image data exerts on a super-resolution image are described next with reference to FIGS. 3A, 3B and 3C. FIG. 3A illustrates an example of a transfer function (MTF) (10) for the label concentration spatial frequency characteristic of the sample S in a spatial frequency space of a super-resolution image, and a spatial frequency characteristic (11) of the noise component. FIG. 3B illustrates an example of image forming patterns (12) of respective spatial frequency components f1, f2 and f3 in a case where no noise component is present. FIG. 3C illustrates an example of image forming patterns (12′) of the respective spatial frequency components in a case where a noise component (13′) is present.
For a transfer of the spatial frequency components from the sample S to the super-resolution image in the super-resolution observation apparatus (1), an image forming contrast of the spatial frequency components is described by the transfer function (MTF) (10). Generally, an image forming contrast of the MTF (10) becomes lower as a spatial frequency increases, and the MTF (10) exceeding the cutoff frequency fc′ of the super-resolution image reduces to 0.
Image forming patterns (12) of the spatial frequency components in a case where no noise component is present are initially described. An amplitude of each of the image forming patterns (12) of each of the spatial frequency components in the case where no noise component is present matches an image forming contrast represented by the MTF (10), and the image forming contrast of the spatial frequencies f1, f2 and f3 becomes lower as the spatial frequencies decreases, as illustrated in FIG. 3B. Since the image forming contrast is present for spatial frequency components lower than fc′, its resolution is represented as fc′.
Image forming patterns of the spatial frequency components in a case where a random noise is present, such as a case where the spatial frequency characteristic (11) of the noise component is widely present in a spatial frequency area as illustrated in FIG. 3A, are described next. If the spatial frequency characteristic (11) of the noise component has an intensity higher than the MTF (10) of the spatial frequency f3, image forming patterns (12′) of components equal to or higher than the spatial frequency f3 are drowned by the noise component (13′) and become invisible as illustrated in FIG. 3C. Therefore, the substantial resolution of the super-resolution image at this time becomes lower than f3.
International Publication Pamphlet No. 2006/109448 discloses a method for relatively reducing influences of a noise component by increasing the number of captured images.